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TITLE OF THE INVENTION 
IMAGE PROCESSING METHOD AND IMAGE PROCESSING SYSTEM 

FIELD OF THE INVENTION 
5 The present invention relates to an image 

processing method and image processing system, which 
converts image data scanned by an image processing 
apparatus such as a copying machine or the like into 
vector data that can be re-used by so-called document 
10 creation application software (e.g.. Word or the like). 

BACKGROUND OF THE INVENTION 

In recent years, amid calls for environmental 

issues, moves towards paperless offices have been 
15 promoted, and various techniques that handle digital 

documents have been proposed. 

For excimple, patent reference 1 (Japanese Patent 

Laid-open No. 2001-358863) describes a technique for 

scanning a paper document by a scanner, converting the 
20 scanned data into a digital document format (e.g., JPEG, 

PDF, or the like), and storing the converted data in 

image storage means . 

Patent reference 2 (Japanese Patent Laid-Open 

No. 8-147445) discloses a document management system 
25 which detects regions of respective properties 

contained in a document image, and manages the document 

as contents for respective regions . 
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Furthermore, patent reference 3 (Japanese Patent 
Laid-open No. 10-285378) discloses the following 
technique. That Is, In a digital multl- function 
peripheral (MFP) (comprising a copy function, scan 
5 function, print function, and the like), it is 

confirmed if a scanned image includes a graphic code 
indicating a page ID, and if the graphic code is found, 
a database is searched for the corresponding page ID. 
If the page ID is found in the database, the currently 

10 scanned image is discarded, print data associated with 
that page ID is read out , and a print image is 
generated and is printed on a paper sheet by a print 
operation. - On the other hand, if no corresponding page 
ID is found in the database, the scanned image is 

15 directly copied onto a paper sheet in a copy mode, or a 
PDL command is appended to the scanned image to convert 
the scanned image into a PDL format, and the converted 
data is transmitted in a facsimile or filing mode. 

With the technique of patent reference 1, an 

20 image scanned by the scanner can be saved as a JPEG 
file or PDF file with a compact information size. 
However, this technique cannot search for a saved file 
based on the printed document. Hence, when print and 
scan processes are repeated, the saved digital document 

25 image gradually deteriorates. 

The technique of patent reference 2 divides an 
image into a plurality of regions and allows these 
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regions to be re-usable for respective contents « 
However, the contents are searched on the basis of a 
user's instruction, and contents to be used are 
determined from the found contents. Hence, upon 
5 generating a document using the stored contents, the 
user must determine contents to be used, thus taking a 
lot of trouble. 

With the technique of patent reference 3, if no 
original digital document corresponding to a paper 
10 . document is found, a PDL command is appended to a 

scanned image to convert that image into a PDL format . 
However , when the . PDL command is merely appended to the 
scanned image to convert that image into the PDL format, 
a relatively large file size is required- 

15 

SUMMARY OF THE INVENTION 
The present Invention has been proposed to solve 
the conventional problems, and has as its objects to 
provide an image processing method and image processing 
20 system, which allow the user to easily re-use digital 

data stored in storage means using a portal sheet of an 
input image . 

In order to achieve the above object, an image 
processing method according to the present invention 
25 comprises a search step of searching storage means for 

original digital data corresponding to each input image, 
a vector izat ion step of converting the input image into 
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vector data when no original digital data Is found In 
the search step, a storage step of storing the Image 
that has been converted Into the vector data in the 
vectorlzatlon step In the storage means as digital data 
5 and a sheet generation step of generating a sheet 

Including at least one of Information associated with 
the found original digital data when the original 
digital data is found in the search step and 
Inforrnation associated with digital data which is 

10 obtained by converting the image into the vector data 
in the vectorlzatlon step and is stored in the storage 
step when no original digital data is found in the 
search step. 

In order to achieve the above object, an image 

15 processing system according to the present invention 
comprises search means for searching storage means for 
original digital data corresponding to each input image, 
vectorlzatlon means for converting the input image into 
vector data when no original digital data is found by 

20 the search means , save meeuis for storing the image that 
has been converted into the vector data by the 
vectorlzatlon means in the storage means as digital 
data and sheet generation means for generating a sheet 
Including at least one of information associated with 

25 the found original digital data when the original 
digital data is found by the search means and 
information associated with digital data which is 
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Obtained by converting the Image into the vector data 
by the vectorization means and is stored by the save 
means when no original digital data is found by the 
search means. 

5 Further objects, features and advantages of the 

present invention will become apparent from the 
following detailed description of embodiments of the 
present invention with reference to the accompanying 
drawings . 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 
The accompanying drawings, which are Incorporates 
in and constitute a part of the specification, 
illustrate embodiments of the invention and, together 
15 with the description, serve to explain the principle of 
the invention. 

Fig. 1 is a block diagram showing an example of 
the arrangement of an image processing system; 

Fig. 2 is a block diagram showing an example of 
20 the arrangement of an MFP; 

Fig. 3 is a flow chart showing an overview of the 
entire operation of an image process; 

Fig, 4 shows an example of a block selection 
process; 

25 Fig. 5 shows an example of the configurations of 

block information and input file information; 
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Fig* 6 Is a flow chart showing a decoding process 
of a two-dimensional barcode; 

Fig . 7 shows an example of a document appended 
with a two-dimensional barcode; 
5 Fig. 8 is a flow chsort of a file search process 

based on pointer information; 

Fig. 9 is a flow chart showing an access -limited 
file search process; 

Figs. lOA and lOB are flow charts showing a 
10 layout search process of a file; 

Fig. 11 is an explanatory view of a vectorization 
process; 

Fig. 12 is an explanatory view of a vectorization 
process; 

15 Fig. 13 is a flow chart showing a grouping 

process of vector data; 

Fig. 14 is a flow chart showing a graphic element 
detection process; 

Fig. 15 shows the data structure of a DAOF; 
20 Fig. 16 is a flow chart showing an application 

data conversion process; 

Fig. 17 is a flow chart showing a document 
structure tree generation process ; 

Figs. 18A and 18B are views for explaining the 
25 document structure tree generation process; 

Fig. 19 shows an ex2unple of a portal sheet; 
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Fig. 20 is a flow chart showing a portal sheet 
generation process; 

Fig. 21 Is a flow chart showing a pointer 
Information encoding process ; 
5 Fig. 22 shows a portal sheet edit window; and . 

Fig. 23 shows a portal sheet edit window upon 
accessing partial Information. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

10 An embodiment of the present Invention will be 

described hereinafter. Fig. 1 Is a block diagram 
showing an example of the arrangement of an Image 
processing system according to the present Invention. 
This Image processing system Is Implemented In an 

15 environment In which offices 10 and 20 are connected 

via an Internet 104. To a LAN 107 formed In the office 
10, an MFP ( Mult 1 -Function Peripheral) 100, a 
management PC 101 for controlling the MFP 100, a client 
PC 102, a document management server 106, Its database 

20 105, and a proxy server 103 are connected. The LAN 107 
and a LAN 108 In the office 20 are connected to the 
Internet 104 via the proxy servers 103- The MFP 100 
has charge of an Image scanning process (scanner) of 
paper documents and some of Image processes for a 

25 scanned Image signal In the present Invention, and 

supplies an Image signal to the management PC 101 via a 
LAN 109. The management PC comprises a general PC, 
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which has Image storage means. Image processing means, 
display means. Input means, and the like, but some 
means of which are Integrally formed with the MFP 100. 

Fig. 2 Is a block diagram showing the arrangement 
5 of the MFP 100. Referring to Fig. 2, an Image scanning 
unit 110 Including an auto document feeder (to be 
abbreviated as an ADF hereinafter) irradiates a 
document Image on each of one or a plurality of stacked 
documents with light coming from a light source (not 

10 shown), forms an image of light reflected by the 

document on a solid-state image sensing element via a 
lens, and obtains a scanned image signal in the raster 
order as Image information at a resolution of, for 
exaimple, 600 DPI, from the solid-state image sensing 

15 element. In a normal copy function, a data processing 
device 115 executes an image process of that image 
signal to convert It into a recording signal. In case 
of a multi-copy process, the data processing device 115 
temporarily stores recording data in a storage device 

20 111, and sequentially outputs that data to a recording 
device 112, thus forming images on paper sheets. 

Print data output from the client PC 102 is 
received by the data processing device 115 via the LAN 
107 and a network I/F 114, and is then converted into 

25 recordable raster data by the data processing device 
115. The raster data is then input to the recording 
device to form a recording image on a paper sheet . 
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The operator Inputs Instructions to the MFP using 
a key console equipped on the MFP, and an Input device 
113 which Includes a keyboard and mouse whose outputs 
are to be Input to the management PC, Such series of 
5 operations are controlled by a controller (not shown) 
In the data processing device 115. 

A display device 116 displays status of operation 
Inputs and image data whose process is underway. Note 
that the storage device 111 is also controlled from the 

10 management PC, and data exchange and control between 

these MFP and management PC are done via a network I/F 
117 and the directly connected LAN 109. 

Note that the present invention is practiced in 
the apparatus shown in Fig. 2 or the system shown in 

15 Fig. 1. For example, the processes of the present 

Invention (to be described later) may be Implemented 
when a controller (CPU) of the data processing device 
115 executes a computer executable control progreim 
stored in the storage device 111 in Fig. 2 (in this 

20 case, the control program that forms the present 

Invention may be stored in the storage device 111 or 
may be loaded and executed from an external device via 
a communication line or the like; the storage device is 
not limited to a built-in hard disk but may be a 

25 removable disk or the like). Also, the process of the 
present invention may be implemented by an electrical 
circuit as hardware components which include all or 
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some coinponents of the controller of the data 
processing device 115. 
<<Overview of Process >> 

An overview of the entire image process according 
5 to the present invention will be described below using 
Fig. 3. 

Referring to Fig. 3, the image scanning unit 110 
of the MFP 100 is enabled to scan one document in a 
raster order, and to obtain an 8-bit image signal of a 

10 600 DPI in an image information input process 120. 

This image signal undergoes a pre-process by the data 
processing device 115, and is stored as image data for 
one page in the storage device 111. A CPU of the 
management PC 101 separates regions of a text /line 

15 image part and halftone image part from the stored 
image signal. The CPU further separates a text part 
into blocks combined as clusters for respective 
paragraphs, and a line image part into tables and 
graphics formed of lines, and converts these tables and 

20 graphics into segments. On the other hand, the CPU 
segments an image part expressed by halftone into 
independent objects for respective so-called blocks 
(e.g., a rectangular image part block, background part 
block, and the like) (step 121). 

25 At this time, ah object corresponding to a 

two-dimensional barcode or URL, which is recorded in 
the document image as additional information, is 
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detected. In this case, the URL undergoes a character 
recognition process (OCR) or a corresponding mark of 
the two-dimensional barcode is decoded (OMR) (step 122), 
thereby detecting pointer Information in the storage 
5 device that stores an original digital file of that 
document (step 123). As a method of appending the 
pointer information, a method of embedding infoannation 
by slightly changing the spacings between neighboring 
characters, a method of embedding information in a 
10 halftone image as a digital watermark, or the like may 
be adopted. 

If the pointer information is detected, the flow 
branches to step 125 to search. for an original digital 
file from the address designated by the pointer. The 

15 digital file is stored in one of a hard disk in the 
client PC, the database 105 managed by the document 
management server 106 connected to the LAN in the 
office 10 or 20, and the storage device 111 of the MFP 
100 itself in Fig. 1, and these storage devices are 

20 searched in accordance with the address information 

obtained in step 123. If no digital file is found or 
if the found file is a so-called image file represented 
by JPEG, PDF, or tiff in step 125, or if the pointer 
information itself is not found in step 124, the flow 

25 branches to step 126. If the digital file is found 

based on the pointer information in step 125, the flow 
branches to step 133. 
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Step 126 Is a so-called dociiment search process 
routine, A full- text search is conducted by extracting 
words from the results of the OCR process which is 
executed for text blocks in step 122, and comparing 
them with words contained in a digital file, or a 
so-called layout search is conducted by comparing the 
layout and properties of respective objects with those 
of a digital file. As a result of the search process, 
if digital files with high similarity levels are found, 
digital files as candidates are displayed as thumbnails 
or the like (step 127), and a file is specified from 
the plurality of candidates by an operator's input 
operation if necessairy. If a digital file is specified, 
the flow jumps to step 133; if no digital file is found 
by the search process in step 126 or if the specified 
digital file is a so-called image file represented by 
PDF, tiff, or the like, the flow branches to step 129. 

Step 129 is a conversion process step from image 
data into vector data, and converts image information 
into a vectorized digital file. For text blocks that 
have undergone the OCR process in step 122, the size, 
style, and font of characters are further recognized to 
convert the text blocks into font data which are 
visually faithful to characters obtained by scanning 
the document. Also, table and graphic blocks formed of 
lines are converted into outline data. Natural image 
blocks such as a photo or the like are processed as an 
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Independent JPEG file as Image data. Sucli 
vectorization processes are executed for respective 
objects, and layout information of the objects is saved. 
In step 130, these objects and layout information are 
5 converted into application data which can be edited by 
a general application. In step 131, the application 
data is stored in the storage device 111 as a digital 
file. Furthermore, in order to allow a direct search 
process of that digital file upon executing a similar 

10 process, index information for such search process is 
generated and is added to a search index file in step 
132. Note that the data may be converted into an rtf 
(Rich Text Format) file that as a versatile file format 
in step 130. Note that the application data file to be 

15 converted is not limited to the rtf format that allows 
object embedding. For example, data may be converted 
into other file formats such as an SVG (Scalable Vector 
Graphics) format and the like. 

Upon completion of the aforementioned processes, 

20 it is checked in step 133 if the page of interest is 

the last page. If No in step 133, the control returns 
to step 120 to repeat the processes for the next page. 

If the page of interest is the last page, the 
control advances to step 134 . 

25 Step 134 is a process as a characteristic feature 

of the present invention. That is, in step 134 an 
index sheet (also referred to as a portal sheet) which 
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contains page Information (e.g., thumbnail, title, 
abstract, keywords, ID, date, author, pointer 
Information to processing results, and the like) of all 
pages specified via the above scan, search, and 
5 vectorlzatlon processes Is generated and output. 
<<Block Selection Process >> 

The block selection process In step 121 will be 
described first. 

In the block selection process. Image data Input 

10 In step 120 (left side of Fig. 4) Is recognized as 
clusters for respective objects, properties 
( text /plcture/photo/llne/ table, etc.) of respective 
blocks are determined, and the image data Is segmented 
Into regions having different properties, as shown In 

15 the right side of Fig. 4. 

One embodiment of the block selection process 
will be described below. 

An Input Image Is blnarlzed to a monochrome Image, 
and a cluster of pixels bounded by an outline of black 

20 pixels Is extracted by outline tracing of black pixels. 
For a cluster of black pixels with a large area, 
outline tracing Is also made for white pixels In the 
cluster to extract clusters of white pixels. 
Furthermore, a cluster of black pixels Is recursively 

25 extracted from the cluster of white pixels with a 
predetermined area or more • 
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The obtained clusters of black pixels cire 
classified Into regions having different properties in 
accordance with their sizes and shapes. For example, a 
pixel cluster which has an aspect ratio close to 1, and 
5 has a size that falls within a predetermined range is 
determined as that corresponding to a character. 
Furthermore, a part where neighboring characters 
regularly line up and can be grouped is determined as a 
text region. Also, a low-profile pixel cluster is 

10 categorized as a line region, a range occupied by black 
pixel clusters that include rectangular white pixel 
clusters which have a predetermined size or more and 
regularly line up is categorized as a table region, a 
region where pixel clusters with indeterminate forms 

15 are distributed is categorized as a photo region, and 
other pixel clusters with an arbitrary shape is 
categorized as a picture region, and so forth. 

Fig. 5 shows block information for respective 
blocks obtained by the block selection process, and 

20 input file information used to manage blocks included 
in an input image. These pieces of information for 
respective blocks are used to execute vectorization or 
to conduct a search, as will be described later. 
<<Detection of Pointer Inf ormation>> 

25 The OCR/OMR process (step 122) for extracting the 

storage location of a file from image information will 
be described below. 
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Fig. 6 ±s a flow chart showing a sequence for 
decoding a two-dimensional barcode (QR code symbol) 
appended Into a document Image, and outputting a data 
character string. Fig. 7 shows an example of a 
5 document 310 appended with a two-dimensional barcode. 

A CPU (not shown) scans an image that represents 
the document 310 stored in a page memory in the data 
processing device 115 to detect the position of a 
predetermined two-dimensional barcode symbol 311 from 

10 the result of the aforementioned block selection 

process. A position detection pattern of a QR code is 
made up of identical position detection element 
patterns, which are located at three out of four 
corners of the symbol (step 300). 

15 Next, format information that neighbors the 

position detection pattern is decoded to obtain an 
error correction level and mask pattern applied to the 
symbol (step 301). 

After a model number of the symbol is determined 

20 (step 302), an encoded region bit pattern is XORed 
using the mask pattern obtained from the format 
Information to release the mask process (step 303). 

Note that a symbol character is read in 
accordance with the layout rule corresponding to the 

25 model so as to decode message data and an error 
correction code word, (step 304). 
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It is detected if a decoded code includes an 
error (step 305). If any error is detected, the flow 
branches to step 306 to correct that error. 

A data code word is divided into segments on the 
5 basis of a mode indicator and character count indicator 
from the error- corrected data (step 307). 

Finally, data characters are decoded on the basis 
of a specification mode, thus output ting the result 
(step 308) . 

10 Note that data to be encoded in the 

two-dimensional barcode represents address information 
of the corresponding file, which is formed of path 
information including a file server name and file name. 
Conversely, the address information may be formed of a 

15 URL to the corresponding file. 

In this embodiment, the document 310 appended 
with the pointer information using the two-dimensional 
barcode has been described. Alternatively, pointer 
information may be recorded using a character string. 

20 In this case, a block of a character string according 
to a predetermined rule (e.g., a text block located at 
a predetermined position) is detected by the above 
block selection process , and characters of the 
character string block that indicates the pointer 

25 information undergo character recognition, thus 

obtaining the address information of the original file. 
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Furthermore, pointer Information can also be 
assigned by embedding Information In the character 
spaclngs by applying Imperceptible modulation to, for 
example, the spaclngs between neighboring characters In 
5 a character string of a text block 312 or 313 of the 
document 310 shown In Fig. 7. For example, when the 
character spaclngs are detected upon executing a 
character recognition process (to be described later), 
pointer Information can be obtained. Also, pointer 
10 Information can be assigned as an Invisible digital 
watermark In a natural Image 314. 
<<Flle Search Based on Pointer Inf ormatlon>> 

The digital file search process based on the 
pointer Information In step 125 In Fig. 3 will be 
15 described below using the flow chart of Fig. 8. 

A file server Is specified based on an address 
Included In the pointer Information (step 400). 

Note that the file server Indicates the client PC . 
102, the document management server 106 that 
20 Incorporates the database 105, or the MFP 100 Itself 
that Incorporates the storage device ill. Also, the 
address Is a URL or path Information Including a server 
name and file name. 

After the file server can be specified, the 
25 address Is transferred to the file server (step 401). 

Upon reception of the address, the file server searches 
for a corresponding file (step 402). If no file is 
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found (No in step 403), the file server sends a message 
that advises accordingly to the MFP. 

On the other hand, if a file is found (Yes in 
step 403), the file address is notified, and is used in 
5 the process in step 134 (step .408). 
<<File Search Process >> 

Details of the file search process in step 126 in 
Fig. 3 will be described below using Figs. 5, lOA and 
lOB. 

10 The process in step 126 is done if an input 

document (input file) has no pointer information in 
step 124, if pointer information is available but no 
digital file is found or if a digital file is an image 
file in step 125, as described above. 

15 Assume that blocks and an input file extracted as 

a result of step 122 have information (block 
information, input file information) shown in Fig. 5. 
As the information contents, a property, coordinate 
position, width and height, and availability of OCR 

20 information are exemplified. The property categorizes 
each block into one of text, line, photo, picture, 
table, and the like. For the sake of simplicity, 
respective blocks are respectively named as blocks 1, 2, 
3, 4, 5, and 6 in ascending order of coordinate X (i.e., 

25 XI < X2 < X3 < X4 < X5 < X6) . The total number of 
blocks indicates that of those included in an input 
file, and is "6" in Figs. lOA and lOB. Figs. lOA and 
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lOB are flow charts of a layout search of files similar 
to an Input file from the database using these pieces 
of information. Assume that files stored In the 
database have the same kinds of Information (block 
5 Information, file Information) as those In Fig. 5. 

In the flow of this flow chart, an Input file Is 
compared In turn with those In the database. In step 
510, similarity levels and the like (to be described 
later) are Initialized. In step 511, the total numbers 

10 of blocks are compared. If Yes In step 511, respective 
pieces of Information of blocks In files are compared 
In turn. Upon comparing Information of blocks, 
property, size, and OCR similarity levels are 
calculated In steps 513, 515, and 518, and a total 

15 similarity level Is calculated based on these levels In 
step . 522. Since a method of calculating each 
similarity level can use a known technique, a 
description thereof will be omitted. If It Is 
determined In step 523 that the total similarity level 

20 Is higher than a pre- set threshold value Th, that file 
Is determined as a similar candidate In step 524. In 
Figs. lOA and lOB, N, W, and H are respectively the 
total number of blocks, the block width, and the block 
height In an Input file, and AN, AW, and AH are values 

25 which consider errors with reference to the block 

Information of the Input file. Also, n, w, and h are 
respectively the total number of blocks, the block 
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width, and the block height In a file stored In the 
database. Although not shown, position information X, 
position Information Y, and the like may be compared 
upon comparing sizes in step 514. 
5 As a result of the above search process database 

files which have similarity levels higher than the 
threshold value Th and are saved as candidates (step 
524) are displayed as thumbnails or the like (step 127), 
If a plurality of candidates are displayed, a file is 
10 specified from these candidates by the operator's input 
operation . 

<<Vectorization Process>> 

The vectorization process in step 129 in Fig. 3 
will be described in detail below. In this process, if 

15 it is determined that no original digital file is 

present in the file server, image data (scanned input 
image data) shown in Fig. 4 is converted into vector 
data for respective blocks. 
<Vectorization of Text Block> 

20 Initially, respective characters in a text block 

undergo a character recognition process. In the 
character recognition process, an image extracted for 
each character is recognized using one of pattern 
matching methods to obtain a corresponding character 

25 code. In this recognition process, an obseirvation 
feature vector obtained by converting a feature 
acquired from a character image into a 
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several-dozens -dimensional niimerlcal value string Is 
compared with a dictionary feature vector obtained In 
advance for each character type, and a character type 
with a shortest distance Is output as a recognition 
5 result. Various known methods are available for 

feature vector extraction. For example, a method of 
dividing a character Into a mesh pattern, and counting 
character lines In respective meshes as line elements 
depending on their directions to obtain a (mesh 

10 count ) -dimensional vector as a feature is known. 

When a text region extracted by block selection 
(step 121) undergoes character recognition, the writing 
direction (horizontal or vertical) is determined for 
that region, lines are extracted in the corresponding 

15 directions, and character images are then obtained by 
extracting characters. Upon determining the writing 
direction (horizontal or vertical), horizontal and 
vertical projections of pixel values in that region are 
calculated, and if the variance of the horizontal 

20 projection is larger than that of the vertical 
projection, that region can be determined as a 
horizontal writing region; otherwise, that region can 
be determined as a vertical writing region. Upon 
decomposition into character strings and characters, in 

25 case of horizontal writing, lines are extracted using 

the horizontal projection, and characters are extracted 
based on the vertical projection for the extracted line. 
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In case of a vertical writing text region, the 
relationship between the horizontal and vertical 
parameters may be exchanged. Note that a cheoracter 
size can be detected at that time. 

Next, the font type of respective characters in 
the text block of interest is recognized. A plurality 
of sets of dictionary feature vectors for the number of 
character types used in character recognition are 
prepared in correspondence with character shape types 
(i.e., font types), and a font type is output together 
with a character code upon matching, thus recognizing 
the font of a character. 

Using a character code and font information 
obtained by the character recognition and font 
recognition, and outline data prepared for each 
character code and font, information of a character 
part is converted into vector data. If an original 
document is a color document , the color of each 
character is extracted from the color image and is 
recorded together with vector data. 

With the above processes, image information which 
belongs to a text block can be converted into vector 
data with a nearly faithful shape, size, and color. 
<Vectorization of Line Image Block, Picture Block, 
EtO 

For a region which is determined as a line image, 
picture, or table region by the block selection process 
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(step 121), outlines of pixel clusters extracted in 
each region of interest are converted into vector data. 
More specifically, a point sequence of pixels which 
form an outline is divided into sections at a point 

. 5 which is considered as a corner, and each section is 
approximated by a partial line or curve . The corner 
means a point corresponding to a maximal cuirvature, and 
the point corresponding to the meocimal curvature is 
obtained as a point where a distance between an 

10 arbitrary point Pi and a chord which is drawn between 
points Pi-k and Pi+k separated k points from the point 
Pi in the left and right directions becomes mciximal, as 
shown in Fig. 11. Furthermore, let R be (chord 
length/arc length between Pi-k and Pi+k) . Then, a 
• 15 point where the value R is equal to or smaller than a 
threshold value can be considered as a corner. 
Sections obtained after division at each corner can be 
vectorized using a method of least squares or the like 
with respect to a point sequence for a line, and a 

20 ternary spline function or the like for a curve. 

When an object has an inside outline, it is 
similarly approximated by a partial line or curve using 
a point sequence of a white pixel outline extracted by 
tracing the outline of white pixels. 

25 As described above, using partial line 

approximation, an outline of a graphic with an 
arbitrary shape can be vectorized. When an original 
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document has a color linage, the color of a graphic Is 
extracted from the color Image and Is recorded together 
with vector data. 

Furthermore, when an outside outline Is close to 
an Inside outline or another outside In a given section 
as shown In Fig. 12, the two outlines may be combined 
to express a line with a given width. More 
specifically, lines are drawn from respective points PI 
on a given outline to points Ql on another outline, 
each of which has a shortest distance from the 
corresponding point. When the distances PlQl maintain 
a constant value or less on the average, the section of 
Interest Is approximated by a line or curve using 
middle points Rl as a point sequence, and the average 
value of the distances PlQl Is set as the width of that 
line or curve . A line or a table ruled line as a set 
of lines can be efficiently vectorized as a set of 
lines having a given width, as described above. 

Note that vectorlzatlon using the character 
recognition process for a text block has been explained. 
A character which has the shortest distance from a 
dictionary as a result of the character recognition 
process is used as a recognition result. When this 
distance is equal to or larger than a predetermined 
value, the recognition result does not always match an 
original character, and a wrong character having a 
similar shape is often recognized. Therefore, in the 
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present invention, such character is handled in the 
same manner as a general line image, as described above, 
and is converted into outline data. That is, even a 
character that causes a recognition error in the 
5 conventional character recognition process can be 

prevented from being vectorized to a wrong character, 
but can be vectorized based on outline data which is 
visually faithful to image data. 

Note that a photo block cannot be vectorized, and 

10 is output as image data in the present invention. 

A process for grouping vectorized partial lines 
for each graphic object after the outline of a graphic 
with an arbitrary shape is vectorized, as described 
above, will be further described below. 

15 Fig. 13 is a flow chart until vector data are 

grouped for each graphic object. Initial and terminal 
points of each vector data are calculated (step 700). 
Using the initial point information and terminal point 
information of respective vectors, a graphic element is 

20 detected (step 701). Detecting a graphic element is to 
detect a closed graphic formed by partial lines. Such 
detection is made by applying the principle that each 
vector which forms a closed shape has vectors coupled 
to its two ends. Next, other graphic elements or 

25 partial lines present in the graphic element are 

grouped to set a single graphic object (step 702). If 
other graphic elements or partial lines are not present 
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In the graphic element, that graphic element is set as 
a graphic object . 

Fig. 14 is a flow chart showing the process for 
detecting a graphic element. Closed graphic forming 
5 vectors are extracted from vector data by excluding 

unwanted vectors, two ends of which are not coupled to 
other vectors (step 710). An initial point of a vector 
of interest of the closed graphic forming vectors is 
set as a start point, and vectors are traced clockwise 

10 in turn. This process is made until the start point is 
reached, and all passing vectors are grouped as a 
closed graphic that forms one graphic element (step 
711). Also, all closed graphic forming vectors present 
in the closed graphic are grouped. Furthermore, an 

15 initial point of. a vector which is not grouped yet is 

set as a start point, and the above process is repeated. 
Finally, of the unwanted vectors excluded in step 710, 
those which join the vectors grouped as the closed 
graphic in step 711 are detected and are grouped as one 

20 graphic element (step 712). 

With the above process , a graphic block can be 
handled as an independently re-usable graphic object. 
<<Conversion Process to Application Data>> 

The results after the vectorization process of 

25 image data for one page (step 129) are converted into a 
file of an intermediate data format., as shown in 
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Fig. 15. Such intermediate data format is called a 
document analysis output format (DAOF) • 

Fig. 15 shows the data structure of the DAOF. 
Referring to Fig. 15, reference numeral 791 denotes a 
5 Header which holds information associated with document 
image data to be processed, A layout description data 
field 792 holds property information and. block address 
information of respective blocks which are recognized 
for respective properties such as TEXT (text), TITLE 

10 (title), CAPTION (caption), LINEART (line image), 

PICTURE (natural image), FRAME (frame), TABLE (table), 
and the like. A character recognition description data 
field 793 holds character recognition results obtained 
by executing character recognition of TEXT blocks such 

15 as TEXT, TITLE, CAPTION, and the like. A table 
description data field 794 stores details of the 
structure of TABLE blocks. An image description data 
field 795 stores image data of PICTURE blocks, LINEART 
blocks, and the like extracted from the document image 

20 data. 

Such DAOF itself is often saved as a file in 
place of intermediate data. However, in the state of a 
file, a general document creation application cannot 
re-use individual objects. Hence, a process for 
25 converting the DAOF into application data (step 130) 
will be described in detail below using Fig. 16. 

In step 8000, DAOF data is input. 
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In step 8002, a document structure tree which 
serves as a basis of application data Is generated. 

In step 8004, actual data In the DAOF are Input 
based on the document structure tree, thus generating 
5 actual application data. 

Fig. 17 Is a detailed flow chart of the document 
structure tree generation process In step 8002, and 
Figs . 18A and 18B are explanatory view of a document 
structure tree. As a basic rule of the overall control, 
10 the flow of processes transits from a mlcroblock 

(single block) to a macroblock (a set of blocks). In 
this case, a block Indicates a mlcroblock and 
macroblock . 

In step 8100, re-grouping Is done for respective 
15 blocks on the basis of relevance In the vertical 
direction. Immediately after the flow starts, 
determination Is made for respective mlcroblocks . Note 
that relevance can be defined by checking If the 
distance between neighboring blocks Is small, blocks 
20 have nearly the same block widths (heights In case of 
the horizontal direction) , and so forth. Information 
of the distances , widths , heights , and the like can be 
extracted with reference to the DAOF. 

Fig. 18A shows an actual page configuration, and 
25 Fig. 18B shows a document structure tree of that page. 
As a result of the process in step 8100, T3 , T4, and T5 
are determined to form one group VI, T6 and T7 are 
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determined to form one group V2, and these groups are 
generated as those which belong to an identical layer. 

In step 8102, the presence/absence of a vertical 
separator is checked. Physically, a separator is an 
5 object which has a line property in the DAOF. 

Logically, a separator is an element which explicitly 
divides blocks in an application. Upon detection of a 
separator, a group is re -divided in the identical layer. 
It is then checked in step 8104 using a group 

10 length if no more divisions are present. If the group 
length in the vertical direction agrees with a page 
height (a distance between the uppermost and lowermost 
ends of a plurality of blocks present in the page of 
interest), the document structure tree generation 

15 process ends. In case of Figs..l8A and 18B, groups VI 
and V2 have no separator, and their group height does 
not agree with the page height. Hence, the flow 
advances to step 8106. 

In step 8106, re-grouping is done for respective 

20 blocks on the basis of relevance in the horizontal 
direction. In this process as well, the first 
determination immediately after the start is done for 
respective microblocks. The definitions of relevance 
and its determination information are the same as those 

25 in the vertical direction. 

In case of Figs. 18A and 18B, Tl and T2 generate 
group HI, and VI and V2 generate group H2. Groups HI 
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and.H2 are generated as those which belong to an 
Identical layer one level higher than VI and V2 . 

In step 8108, the presence/absence of a separator 
In the horizontal direction is checked. Since 
5 Figs. 18A and 18B Include separator SI, that separator 
is registered in a tree, thus generating layers HI, SI, 
and H2. 

It is checked in step 8110 using a group length 
if no more divisions are present. When the group 
10 length in the horizontal direction agrees with a page 
width, the document structure tree generation process 
ends. 

Otherwise, the flow returns to step 8102 to 
repeat the processes from relevance check in the 
15 vertical direction in an upper layer by one level. 

In case of Figs. 18A and 18B, since the group 
length agrees with the page width (a distance between 
the leftmost and rightmost ends of a plurality of 
blocks present in the page of interest), the process 
20 ends, and uppermost layer VO that represents the entire 
page is finally appended to the document structure tree. 

After the document structure tree is completed, 
application data is generated based on that information 
in step 8004. A practical example in case of Figs. 18A 
25 and 18B will be explained below. 

That is, since HI includes two blocks Tl and T2 
in the horizontal direction, it is output as two 
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columns. After Internal Information of Tl (with 
reference to the DAOF, text as the character 
recognition result, image, and the like) is output, a 
new column is set, and Internal infojnnation of T2 is 
5 output. After that, separator SI is output. 

Since H2 includes two blocks VI and V2 in the 
horizontal direction, it is output as two columns. 
Internal information of VI is output in the order of T3, 
T4, and T5, and a new column is set. Then, internal 
10 information of V2 is output in the order of T6 and T7. 

In this way, since the conversion process into 
application data can be done in the output order, for 
exsunple, the converted application data can have a 
correct read order of text regions and the like. 
15 <<Generation of Portal Sheet >> 

A portal sheet 2201 will be explained first with 
referencie to Fig. 19. The portal sheet shown in 
Fig. 19 records information (information associated 
with an image input in step 120) of six documents. 
20 Information of each document includes the following 
four pieces of information 2202 to 2205. 

Reference numeral 2203 denotes a thumbnail image 
(thumbnail) of a document, which represents the first 
page. 

25 Reference numerals 2202 and 2204 denote a 

plurality of pieces of information associated with the 
thumbnail 2203, which include a date, title, author, ID, 
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keyboard, abstract, and the like. These pieces of 
information Include those which are appended in advance 
to the document, are input by the user, or are 
automatically extracted from the contents of the 
5 document . 

Reference numeral 2205 denotes a pointer 
indicating the storage location or the like of the 
document. In this embodiment, the pointer 2205 is 
expressed by a two -dimensional barcode. Alternatively, 
10 a linear barcode or digital watermark may be used. 

A portal sheet generation method corresponding to 
the process in step 134 in Fig. 3 will be described 
below using Fig. 20. 

The operator selects a portal sheet output mode 
15 from a control panel (not shown) in advance, and inputs 
the number of cases (the number of documents per page) 
to be displayed per sheet using a ten -key pad of the 
control panel. In this example, assume that "6" is set. 

It is confirmed in step 2211 if a document is 
20 laid out in a page of a portal sheet. Since no 

document is laid out Initially, an initialization 
process of the portal sheet page is executed in step 
2212. 

. A set of document Information which includes a 
25 date, title, and the like of each document is acquired 
(2213), and the acquired information is laid out on the 
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portal sheet page (2214), as shown in Fig. 19. In step 
2214, pointer information of each document is appended. 

It is confirmed if the number of documents laid 
out in the portal sheet page reaches the number of 
5 documents per page designated by the operator (2215). 
If Yes in step 2215, the current page is output to the 
printer. 

Finally, it is confirmed if all documents have 

been laid out (2217). If a document to be laid out is 
10 found, the flow returns to step 2211; otherwise, the 

last page is output as needed (2218, 2219), thus ending 

the process . 

A method of generating a set of document 

information (title, keywords, ID, date, author, 
15 abstract, table of contents, and the like) which is 

acquired in step 2213 and is laid out in step 2214 will 

be explained below. 

As a method of extracting a title from text data 

contained in a document, for excunple, a sentence which 
20 appears at the start of the text data, has a large font 

size, and has a high frequency of use in the text data 

is extracted as a title using the layout information 

and font information extracted by the aforementioned 

method . 

25 As a method of extracting keywords from text data 

contained in a document, for example, the entire text 
data is decomposed into words by, among others. 
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morphological analysis. All words are sorted in 
accordance with their frequencies of use, and are 
selected as keywords in descending order of frequency 
of use. In order to extract more effective keywords, 
5 words may be compared with a keyword database, which is 
prepared in advance. 

As for information of an ID, date, and author, if 
a file is found by a digital file search process, such 
infoarmation is acquired as property information of that 
10 file. 

As for abstract information, the following method 
of generating an abstract of text data formed of a 
character code sequence may be used. Text data is 
decomposed into sentences or clauses. In this case, 

15 the text data can be automatically divided into 
sentences or clauses using character codes of 
punctuation marks as boundaries. Next, an importance 
level is calculated for each sentence or clause. As a 
method of calculating the importance level, a method of 

20 calculating the frequencies of occurrence of words 

contained in the entire text data, giving a high score 
to a word that appears frequently, and calculating the 
importance level of each sentence or clause as a sum 
total of the scores of words included in that sentence 

25 or clause, a method of calculating the importance level 
by detecting a sentence corresponding to a title or 
headline using layout information and font information 
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to increase the Importance level of that sentence, or 
to increase the scores of words included in that 
sentence, and the like may be used. Finally, an 
abstract is generated by extracting sentences or 
5 clauses with higher importance levels. By changing the 
number of importance levels to be extracted, an 
abstract with a desired size can be generated. 

As for generation of table of contents 
information, for example, when a table of contents is 

10 to be generated for text data for a plurality of pages, 
text is divided into paragraphs based on spaces on a 
layout, and these paragraphs undergo an abstract 
generation process to assign abstract sentences, each 
for about one line. Especially, sentences which are 

15 extracted as a title or headlines based on the layout 
information and font information are best suited to a 
table of contents . 

A pointer information appending process for 
appending pointer information (e.g., 2205 in Fig. 19) 

20 to indicate the storage location of a document will be 
explained below. Assume that this pointer information 
appending process is executed upon laying out document 
information in step 2214. 

Fig. 21 is a flow chart showing the sequence for 

25 encoding a data character string as pointer information 
by a two-dimensional barcode (QR code symbol: JIS 
X0510). 
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Data to be encoded ±n the two-dimensional barcode 
represents address Information of the corresponding 
file, which Is formed of, for example, path Information 
Including a file server name and. file ncune. Or the 
5 address Information may be formed of a URL to the 

corresponding file, a file ID managed In the database 
105 that stores the corresponding file or In a storage 
device of the MFP 100 Itself, or the like. 

In order to Identify different types of 

10 characters to be encoded, an Input data sequence Is 

analyzed. Also, error detection and error correction 
levels are selected, and a mlnlmxim model number that 
can store Input data Is selected (step 900). 

The Input data sequence Is converted Into a 

15 predeteirmlned bit sequence, and an Indicator Indicating 
a mode (numeric, alphanumeric, 8blt • byte, kanjl, etc.) 
and an end pattern are appended as needed. Furthermore, 
the bit sequence Is converted Into predetermined bit 
code words (step 901). 

20 At this time, for the purpose of error correction, 

the code word sequence Is segmented Into a 
predetermined number of blocks In accordance with the 
model number and error correction level, and error 
correction code words are generated for respective 
.25 blocks and are appended after the data code word 
sequence (step 902). 
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The data code words of respective blocks obtained 
in step 902 are connected, and error correction code 
words and remainder code words as needed are connected 
after the data code word sequence (step 903). 
5 Next, the code word module is set in a matrix 

together with a position detection pattern, separation 
pattern, tdLming pattern, alignment pattern, and the 
like (step 904) . 

Furthermore, a mask pattern optimal to the symbol 
10 encoding region is selected, and is converted by- 
calculating XORs with the module obtained. in step 904 
(step 905) . 

Finally, type information and model number 
information are generated for the module obtained in 
15 step 905, thus completing a two-dimensional code symbol 
(step 906). 

The aforementioned two-dimensional barcode that 
incorporates address information is appended upon 
laying out document information in step 2214. For 

20 example, when a portal sheet is generated from the 
client PC 102, the two-dimensional barcode may be 
converted into recordable raster data by the data 
processing device 115, and may be appended to a 
predetermined position on raster data upon image 

25 formation. 

As means for appending additional information, a 
method of directly appending pointer information to a 
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document as a character string, and so-called 
watermarking methods Including a method o£ embedding 
Information by modulating the spaclngs of a character 
string In a docioment (especially, the spaclngs between 
5 neighboring characters), a method of embedding 

Information In a halftone Image In a document, and the 
like, can be applied In addition to the two-dimensional 
barcode described In this embodiment. 

An example of embedding code Information In a 

10 halftone Image (thumbnail Image) placed on the portal 

sheet using a digital watermark will be explained below. 

Code Information Is embedded as a binary data 
sequence by weighting one bit per pixel of each color 
component on an Image. Upon embedding code Information 

15 In pixels, one code Information Is embedded In one 
block formed of neighboring pixels. The code 
Information Is repetitively embedded In a plurality of 
blocks which do not overlap each other. A block formed 
by a plurality of blocks embedded with identical code 

20 information will be referred to as a macroblock 

hereinafter. Note that the entire thumbnail image can 
be expressed by a macroblock. Upon extracting embedded 
code information, cross -correlation values for 
respective blocks are calculated from the macroblock, 

25 and code information can be extracted based on the 

statistical amount of these cross -correlation values. 
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Code information can be sufficiently embedded in 
an image by changing pixels to the level at which it is 
imperceptible to a human being. However, the image 
quality of a thumbnail image used on the portal sheet 
5 is not so important. Hence, robust information can be 
embedded using larger weights. 
<<Edit Process of Portal Sheet >> 

The portal sheet prepared as described above will 
be explained below. 

10 A desired edit process of a portal sheet required 

to access a designated page, a designated paragraph, a 
designated image, the designated page of a designated 
document, the designated paragraph of the designated 
page of the designated document, the designated image 

15 of the designated page of the designated document, and 
the like using the scanned portal sheet will be 
explained below using Figs. 22 and 23. Note that an 
embodiment to be described below is a fundamental form, 
and can be arbitrarily modified. The edit process of a 

20 portal sheet obtained by scanning the portal sheet will 
be explained. Alternatively, the portal sheet prepared 
as described above may be directly held as digital data, 
and the held portal sheet may be edited. 

Fig. 22 is a schematic view of a user interface 

25 (to be abbreviated as UI hereinafter) 2501 of the MFP 
or the like. As described above, since information of 
a thumbnail, title, date, abstract, keywords, pointer 
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Information, and the like can be obtained from a 
digital watermark or the like by scanning the portal 
sheet , they are displayed on the UI , and the user edits 
a desired new portal sheet by observing such 
5 inf orma t ion . 

Respective parts displayed on the UI will be 
described below. 

Reference numerals 2502 to 2506 denote buttons 
which are used to edit the portal sheet using the 

10 displayed thumbnail, and respectively have functions of 
"add", "delete", "select all", "access to partial 
information", "end", and the like in the order named. 
Reference numerals 2530 and 2531 denotes buttons which 
have functions of "back to previous page" and "go to 

15 next page" used when a plurality of portal sheets are 

scanned. Reference numerals 2510 to 2515 denote fields 
that display information each including a thumbnail, 
title, date, and the like obtained after the portal 
sheet is scanned. Reference numerals 2520 to 2525 

20 denote information fields on an edited portal sheet (to 
be referred to as a new portal sheet hereinafter). 

The edit operation of the portal sheet in Fig. 22 
will be described below. The user touches a desired 
thumbnail on the left area of the window with a finger 

25 or pen to activate that thumbnail, and then presses the 
add button 2502, thus adding the thumbnail to the right 
area of the window (new portal sheet window) . By 
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pressing the select all button 2504, all thumbnails 
displayed on the left area of the window can be 
activated. A thumbnail to be deleted from the new 
portal sheet can be deleted by pressing the delete 
5 button 2503. When the user wants to display the next 
scanned portal sheet on the left area of the window, he 
or she presses the page change button 2530 or .2531 • 
Upon completion of the desired edit operation, the user 
quits the operation by pressing the "end" button 2506. 

10 Fig. 22 shows the result obtained after the user 

selects (activates) the thumbnails 2511 and 2515 and 
then presses the add button. Note that the portal 
sheet can be edited by drag & drop operations in place 
of the operations using the buttons. More specifically, 

15 the user drags a desired thumbnail on the scanned 
portal sheet window while touching it, and drops 
(releases) that thumbnail at a desired add position on 
the new portal sheet window, thus adding that thumbnail. 
When another thumbnail has already been registered at 

20 the add position where the user dropped the thumbnail, 
the already registered thumbnails are shifted in turn 
to their next positions, and the dropped thumbnail is 
added to that add position. When the user touches a 
desired portal sheet on the new portal sheet window, 

25 and drags and drops that sheet at a position outside 
the new portal sheet window, he or she can delete the 
portal sheet from the new portal sheet. Also, the user 
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can change the order of thumbnails registered on the 
new portal sheet by drag & drop operations . For 
example, when the user drags thumbnail F and drops It 
at the position 2520 on the new portal sheet window, 
5 thumbnail B Is displayed at the position 2521 since Its 
registration order Is shifted, and thumbnail F Is 
displayed at the position 2520. 

Although not shown. Information such as a title 
and the like can also be edited . The edit result on 

10 the right area of the window Inherits Inner Information 
(pointer Information to an original Image using a 
digital watermark or the like) of the portal sheet on 
the left area of the window, and Is generated as a new 
portal sheet. In this way, a new portal sheet that 

15 collects only necessary Information can be generated. 
Fig. 23 shows a UI window displayed upon 
selection of the "access to partial Information" button 
(2505) In Fig. 22. A left area 2630 of the window 
displays original digital data (data for each document 

20 page) on the basis of the pointer Information of a 

thumbnail selected from the portal sheet In Fig. 22. 
This original digital data Is segmented Into regions 
based on properties such as a title, body text. 
Illustration, and the like, and objects for respective 

25 segmented regions are Identlflably displayed (2631, 

2632). Fig. 23 provides the UI window that allows the 
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user to add these objects to the portal sheet shown in 
Fig. 22. 

Reference numeral 2633 denotes a field that 
displays the page number of the currently displayed 
5 thumbnail, and another page can be selected using 

buttons 2634 and 2635. Reference numerals 2643 to 2646 
denote fields used to display objects selected from the 
original digital data displayed on the left area. The 
displayed objects can be scrolled using buttons 2640 

10 and 2641. Object add/delete functions and an edit end 
function are implemented using buttons 2650 to 2652. 
Also, drag & drop edit operations (add, delete, order 
change, and the like of objects) can be made in the 
same manner as in Fig. 22. 

15 The process for displaying information obtained 

by scanning a portal sheet on the UI as thumbnails, as 
shown in Fig. 22, and generating a new portal sheet 
using partial information (designated page, designated 
paragraph, designated image, and the like) of documents 

20 displayed as thumbnails will be described in detail 
below. The user selects one, partial information of 
which he or she wants to access, of thumbnails (2510 to 
2515) obtained by scanning the portal sheet, and 
selects the "access to partial information" button 

25 (2505). Then, the UI window shown in Fig. 23 is 

diisplayed, and original digital data of the selected 
thumbnail is displayed on the area 2630. The area 2630 
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allows the user to recognize regions segmented for 
respective vectorized objects (paragraph. Image, and 
the like) at a glance. When the user touches and 
selects partial Information that he or she wants to 
5 access (use), and presses the "add" button, the partial 
Information (object) to be selected can be added to the 
right area of the window. When the user wants to 
access a designated page, he or she can turn the page 
using the buttons 2634 and 2635. Also, the added 

10 partial Information can be deleted by the "delete" 

button. Upon depression of the "end" button, access to 
partial information ends, and the UI shown in Fig, 22 
is displayed again. Then, a plurality of pieces of 
partial information (objects displayed on the right 

15 area of the window in Fig. 23) selected on Fig. 23 are 
added in turn to the new portal sheet (fields 2520 to 
2525) in Fig. 22. In this way, using the "access to 
partial Information" button, the user can add each 
partial information (designated page, designated 

20 paragraph, designated image, or the like) to the new 
portal sheet, i.e., he or she can use only required 
information on the portal sheet . 
<<Use of Portal Sheet>> 

The user can easily select a page or object to be 

25 browsed, transmitted, or printed using the portal sheet 
which is prepared or edited, as described above. For 
example, when a portal sheet is scanned, the scanned 
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portal sheet is displayed, as shovm on the left side of 
Fig. 22. When the user selects a page (or object) to 
be processed from that portal sheet, selects a desired 
processing mode, and then presses a "START" button, 
5 original digital data is acquired based on pointer 
information of the selected page (or object), and 
undergoes a desired process (browse, transmission, or 
print process). 
(Other Embodiments) 

10 < Another Embodiment Associated With File Access Right > 
Re-use of some of our docvmient files, by a third 
party must often be limited. The above embodiment has 
been explained under the condition that all files 
stored in the file server can be freely accessed, and 

15 the entire file or some objects of the file are 

re-usable. Hence, another embodiment associated with a 
case wherein when a file is searched for using pointer 
information in the above embodiment, the access right 
of a file that can be specified as a result of the 

20 search is limited will be described below using Fig. 9. 
Since steps 400 to 403 are the same as those in the 
above embodiment, a description thereof will be omitted. 
If a file is specified in step 403, the file server 
checks access right information of that file in step 

25 404. If the access right of that file is limited, the 
file server requests the MFP to transmit a password 
(step 405) . 
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The MFP prompts the operator to input a password, 
and transmits the Input password to the f±le server 
(step 406) • 

The file server collates the received password to 
5 make authentication (step 407). If authentication has 
succeeded, the server notifies the file address (step 
134), as has been explained using Fig. 3, and transfers 
a file to the MFP if the process that the user wants is 
acquisition of image file data (step 408). 

10 Note that the authentication method for access 

right control is not limited to the method using the 
password in steps 405 and 406. For -example, various 
other authentication means such as popular biological 
authentication (e.g., fingerprint authentication), 

15 authentication using a card, and the like can be 
adopted . 

In this embodiment, a file is specified by 
pointer information additionally given to a paper 
document . The same control can be applied to a case 

20 wherein a file is specified by the search process in 
steps 126 to 128 in Fig. 3. 

Also, the vectorization process explained in 
steps 129 in Fig. 3 can be limited. That is, when a 
limitation of the access right to a given paper 

25 document is detected based on a watermark or the like 
from image information obtained by scanning that paper 
document, the vectorization process is executed only 
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when authentication is successful, thus limiting use of 
documents with high secrecy. 

<Another Embodiment Associated with File Specif ication> 
In the above embodiment, means for specifying 
5 original filie data from image information obtained by 
sccinning a document specifies a document based on 
pointer information appended to the document or 
searches for a corresponding digital file on the basis 
of each object information described in the document, 

10 as has been described using Fig. 3. In order to 

specify an original file more accurately, a digital 
file that meets both of these means can be specified. 
That is, even when the presence of an original file can 
be detected based on pointer information obtained from 

15 a document, a layout search according to layout 

information or a full -text search based on a keyword 
obtained by character recognition is also conducted for 
the detected file using object information in the 
document, and a file that can yield a high matching 

20 rate is formally specified as an original file. For 
example, even when the lower-bit part of pointer 
information is doubtful and cannot be corrected by 
error correction, a file can be specified by narrowing 
down the search range. Therefore, a file can be 

25 quickly and accurately specified. 

<Another Embodiment of Vectorization> 
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In the above embodiment, when search means cannot 
specify an original file, the entire document image 
undergoes a vectorlzatlon process. For example, in 
case of a general document, not all objects in the 
5 document are newly created, and some objects may be 
diverted from other files. For example, a document 
creation application prepares some patterns of 
background objects (wallpapers), and the user normally 
selects and uses one of these patterns. Hence, such 
10 object is more likely to be present in other document 

files in the document file database as re-usable vector 
data. 

Hence, as another embodiment of the vectorization 
process in step 129 in Fig. 3, the database is searched 

15 for a file which includes an object which substantially 
matches each of individual objects segmented by the 
block selection process, and vector data of the matched 
object is individually acquired from that file. As a 
result, since vectorization of the whole document need 

20 not be done, it can be done more quickly, and 

deterioration of image quality due to vectorization can 
be prevented. 

On the other hand, in Fig. 3, when an original 
file can be specified as a PDF in the search process in 

25 steps 126 to 128, that PDF often has, as an additional 
file, character codes obtained by already applying 
character recognition to text objects in that document. 
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Upon vectorizing such PDF file, if the character code 
file is used, a character recognition process in the 
vectorization process in step 129 and subsequent steps 
can be skipped. That is, the vectorization process can 
5 be done more quickly. 

In the above embodiment, each text block is 
converted into vector data on the basis of character 
codes and font data obtained as the character 
recognition result of that text block. Alternatively, 
10 as in the vectorization process of line image blocks 

and picture blocks, outlines of character images may be 
extracted and may be converted into vector data. 
< Another Embodiment of Object Used to Form Portal 
Sheet > 

15 In the above embodiment, as a document to be 

controlled is converted into a portal sheet for 
respective pages of document images. However, the 
present invention is not limited to this , and a portal 
sheet may be generated for respective pages of a 

20 plurality of documents, or may be generated for 

respective documents by displaying their cover pages or 
the like as thumbnails . Also , a portal sheet may 
. include both information for respective documents and 
pages . 

25 Note that the present invention can be applied to 

an apparatus comprising a single device or to system 
constituted by a plurality of devices . 
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Furthermore, the invention can be implemented by 
supplying a software program, which implements the 
functions of the foregoing embodiments, directly or 
indirectly to a system or apparatus, reading the 
5 supplied program code with a computer of the system or 
apparatus , and then executing the program code . In 
this case, so long as the system or apparatus has the 
functions of the program, the mode of implementation 
need not rely upon a program. 

10 Accordingly, since the functions of the present 

invention are implemented by computer, the program code 
installed in the computer also implements the present 
invention. In other words, the claims of the present 
invention also cover a computer program for the purpose 

15 of implementing the functions of the present invention. 

In this case, so long as the system or apparatus 
has the functions of the program, the program may be 
executed in any form, such as an object code, a program 
executed by an interpreter, or scrip data supplied to 

20 an operating system. 

Example of storage media that can be used for 
supplying the program are a floppy disk, a hard disk, 
an optical disk, a magneto- optical disk, a CD-ROM, a 
CD-R, a CD-RW, a magnetic tape, a non-volatile type 

25 memory card, a ROM, and a DVD (DVD-ROM and a DVD-R). 

As for the method of supplying the program, a 
client computer can be connected to a website on the 
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Internet using a browser of the client computer, and 
the computer program of the present Invention or an 
automatically- Installable compressed file of the 
program can be downloaded to a recording medium such as 
a hard disk. Further, the program of the present 
invention can be supplied by dividing the progrsun code 
constituting the program into a plurality of files and 
downloading the files from different websites • In 
other words, a WWW (World Wide Web) server that 
downloads, to multiple users, the program files that 
implement the functions of the present invention by 
computer is also covered by the claims of the present 
invention . 

It is also possible to encrypt and store the 
program of the present invention on a storage medium 
such as a CD-ROM, distribute the storage medium to 
users, allow users who meet certain requirements to 
download decryption key infoarmation from a website via 
the Internet, and allow these users to decrypt the 
encrypted program by using the key information, whereby 
the program is installed in the user computer. 

Besides the cases where the aforementioned 
functions according to the embodiments are implemented 
by executing the read program by computer, an operating 
system or the like running on the computer may perform 
all or a part of the actual processing so that the 
functions of the foregoing embodiments can be 



- 52 - 



CFM03463 / P204-0022 



Implemented by this processing. 

Furthermore, after the program read from the 
storage medium Is written to a function expansion board 
Inserted Into the computer or to a memory provided In a 
function expansion unit connected to the computer, a 
CPU or the like mounted on the function expansion board 
or function expansion unit performs all or a part of 
the actual processing so that the functions of the 
foregoing embodiments can be Implemented by this 
processing. 

As many apparently widely different embodiments 
of the present Invention can be made without departing 
from the spirit and scope thereof. It Is to be 
understood that the Invention Is not limited to the 
specific embodiments thereof except as defined In the 
appended claims . 

As described above, according to the present 
Invention, since a portal sheet of an Input Image is 
generated, digital data stored in storage means can be 
easily re-used using that portal sheet. 

Since pointer Information used to specify 
original digital data Is appended. It Is easy to search 
for the original digital data. 

Since the portal sheet can be edited for 
respective objects (blocks), only required objects can 
be handled, and a process such as a browse process or 
the like can be easily selected. 
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Since vector data are stored, the required 
storage capacity can be reduced. If original digital 
data is found, that original digital data is used, thus 
minimizing deterioration of image quality. 
5 The present invention is not limited to the above 

embodiments and various changes and modification can be 
made within the spirit and scope of the present 
inventions. Therefore, to apprise the public of the 
scope of the present invention, the following claims 
10 are made. 
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